Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record and log additional metrics for O #1670

Merged
merged 5 commits into from
Nov 27, 2020
Merged

Record and log additional metrics for O #1670

merged 5 commits into from
Nov 27, 2020

Conversation

yondonfu
Copy link
Member

@yondonfu yondonfu commented Nov 26, 2020

What does this pull request do? Explain your changes. (required)

Record and log the following metrics on O:

  • Source segment duration
  • Download time for source segment
  • Transcode score for a segment which is defined as source segment duration / transcode time (this is added for standalone T as well)

B downloads the transcoded results from O separately and the associated HTTP handler lives in LPMS so I did not try to add metrics instrumentation there for now. As a result, this PR does not include the ability to measure the ratio between source segment duration and round trip time from O's POV.

A few other minor updates:

  • 1702ba5 uses a NodeType type instead of a string in the monitor package. I felt that this would be less prone to error when writing conditionals based on the current node type as is done in 7a05697 which skips extra state tracking code in SegmentEmerged if the function is used for a node type other than a broadcaster
  • 321f5d0 tweaks the view distribution for the source segment duration metric to make the percentile graph of source segment duration easier to understand

Specific updates (required)

See commit history.

How did you test each of these updates (required)

Tested manually and observed metrics using livepeer/docker-livepeer#44.

Does this pull request close any open issues?

Fixes #1661

Checklist:

  • README and other documentation updated
  • Node runs in OSX and devenv
  • All tests in ./test.sh pass

@yondonfu yondonfu changed the title Yf/orch metrics Record and log additional metrics for O Nov 26, 2020
Copy link
Contributor

@darkdarkdragon darkdarkdragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small change
and not for the future: we're using dur= in logs both for segment's duration and network transfer duration. I think it would be good to separate them, use segDur= for example for segment's duration (not in this PR of course).

server/segment_rpc.go Outdated Show resolved Hide resolved
@yondonfu
Copy link
Member Author

Fixed a monitor package test in 58ffeed that was failing due to the broadcaster only conditional check in SegmentEmerged.

Copy link
Contributor

@darkdarkdragon darkdarkdragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yondonfu yondonfu merged commit 7f9534c into master Nov 27, 2020
@yondonfu yondonfu deleted the yf/orch-metrics branch November 27, 2020 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add logs for and metrics for real-time ratio and segment duration on O/T
2 participants